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Abstract 



, If a fourth generation quark exists whose mass is below 255 GeV, then the 
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^) , pressed. For this reason, CDF has searched for the one-foop neutral current 

, decay b' ^ b + Z, assuming that the branching ratio into b + Z is 100%; an 



only two-body charged current decay, 6' — > c + W, is doubly-Cabibbo sup- 



On, analysis giving the bounds on mi,/ for smaller branching ratios is in prepara- 



D , tion. In this Brief Report, we examine the neutral current decay b' ^ b-\- H, 



which will occur if the Higgs mass is less than my— mi,. Four different cases are 
examined: the sequential case, the non-chiral isosinglet case, the non-chiral 
isodoublet case, and a two-Higgs model with flavor-changing neutral currents. 
In the first three of these, the rates for b' ^ b-\- Z and b' ^ b + H are compa- 
rable, assuming comparable phase space factors; in the fourth, b' ^ b + H is 
completely dominant. Thus, we emphasize the importance of giving b' mass 
bounds as a function of the branching ratio into b + Z, since the assumption 
of a 100% branching ratio for b' ^ b + Z may only be valid if the Higgs mass 
is near or above the b' mass. 
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The CDF Collaboration reported last year a search for the neutral current decay of 
the fourth generation b' quark into b + Z. An earlier result from the DO Collaboration 
ruled out b' masses up to 95 GeV from 6' ^ 6 + 7; above that mass, b' —>■ b + Z dominates. 
The CDF bound is more stringent. In Ref. they have excluded b' quarks with masses up 
to 148 GeV, depending on the b' lifetime, and assuming that the branching ratio of b' b+Z 
is 100%. CDF has presented preliminary limits on the b' production cross-section times 
{BR{b' — s> 6 + Z))^ as a function of the b' mass, and has excluded b' masses from 100 GeV 
up to 170 GeV, for BR{b' b + Z) > 71%. 

It might seem surprising that the neutral current decay, which occurs through one-loop 
in the standard model, could dominate the tree-level charged current decay. However the 
decay b' t + W is forbidden for b' masses below 255 GeV (and the three body phase space 
severely suppresses the decay into t + W* for b' masses below about 230 GeV), and the 
decay b' ^ c+W is doubly-Cabibbo-suppressed. If the mixing angle which connects across 
two generations is very small, which would not be surprising, then the decay b' —>■ b + Z 
could very well be dominant. If the b' quark is non-chiral, either an isosinglet or part of 
an isodoublet, then the GIM violation will lead to a tree- level b' ^ b + Z decay. In that 
case, the neutral current decay will certainly be dominant. A detailed discussion of these 
possibilities, including a full set of formulae and plots, can be found in Ref. [§] . In the early 
work of Mukhopadhyaya and Roy 0, the neutral current decay of sequential, isosinglet, 
isodoublet and mirror quarks were considered. Their bounds were below the Z mass, and 
thus the primary decay modes were into photons and virtual Z's. 

In this Brief Report, we look at a decay mode not considered in the above, b' ^ b + H, 
where H is the Higgs boson. This occurs in the standard model at one-loop, if kinematically 
accessible, and in non-chiral models at tree-level. It is, of course, much more difficult to 
detect, since the H will decay into bb, leading to a purely hadronic signature (although 
see the discussion below). However, it could suppress the overall b' ^ b + Z branching 
ratio, weakening the mass bounds, even in the non-chiral case. We will look at four models: 
the standard model with a sequential b' quark, a vectorlike isosinglet model, a vectorlike 
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isodoublet model, and a two-Higgs model with tree-level flavor changing neutral currents. 
The discussion of 6' ^ 6 + if in the first two of these models is not new; extensive discussions 
have been previously published. However these publications are ten years old, and we now 
know the top mass, precision electroweak studies give constraints, and the CKM angles are 
better known. The decay in the latter two models has not been discussed. 



1. Sequential Quarks 

The simplest realization of a fourth family is to add left-handed doublets and right- 
handed singlets (with a right-handed neutrino necessary to give the extra neutrino a large 
mass). The first calculations oih' h + H were carried out by Hou and Stuart [|| and by 
Eilam, Haeri and Soni ||^. A much more detailed analysis, which was the first to directly 
compare the rate with that of 6' ^ 6 + Z, which made no assumptions about mixing angles 
and which discussed the anomalous thresholds that occur in the calculation, appeared in 
the subsequent work of Hou and Stuart . 

First, consider the ratio of the neutral current decay 6' ^ 6 + Z to the charged current 
decay h' c + W. The former decay depends on the mass of the t' quark and \Vtb'\; the 
latter depends on \Vcb'\- For a t' mass of 250 GeV, the ratio is given by (see Ref [§] for full 
expressions and a plot) 

Tib' ^ bz) „„„,|v;„f 

For different t' masses, the ratio varies roughly as {mf, — mf)'^ (note the GIM cancellation 
when the masses are equal). We thus see how sensitive the ratio is to the mixing angles. 
If one were to choose | Vtf,' | / 1 V^fe/ 1 to be the same as iKbl/lKtl = 13 ± 3, then the above 
ratio is between 0.5 and 1.3. However, the large top quark mass might indicate a very 
large mixing angle between the third and fourth generations, leading to a much bigger ratio. 
Thus, the neutral current h' h + Z decay is certainly similar to, and could dominate, the 
charged- current decay. 
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In the ratio of b' b + H to b' b + Z , the mixing angles cancel, so there is less 
arbitrariness in the result. The result, given by Hou and Stuart [Q, is a function of Mh, 
rrit, rrit' and rrih'. Hou and Stuart give plots of the partial widths as a function of rrib/, for 
four different values of Mh, three different values of rrit and two different values of mf. 
Fortunately, one of the choices for rrit was 175 GeV (the others were 75 and 125 GeV), and 
the dependence on rrif, while important for the individual rates, is very weak in the ratio. For 
rriH = 100 GeV, the ratio oi b' ^ b + H to b' b + Z is approximately (1.0, 1.4, 1.7,2.0,2.5) 
for rrib' = (150, 175, 200, 225, 250), respectively. For rrin = 150 GeV, phase space suppression 
sets in, and the ratio, for the same b' masses, is (0, 0.15, 0.7, 1.0, 1.6), respectively. One sees 
that the two rates are very similar. For a Higgs mass of 100 GeV, and a sequential b' quark, 
the assumption that the branching ratio for b' b + Z is 100% is not valid. On the other 
hand, for a Higgs mass of 150 GeV or higher, it may be reasonable. 

Could one improve upon Hou and Stuart's calculation? We now know the top quark 
mass (and can distinguish between the Yukawa coupling MS mass and the pole mass), we 
know from precision electroweak data that the t' mass cannot be much bigger than the 
b' mass, we have a much better understanding of the production cross sections for heavy 
quarks, and b-tagging in hadron colliders is much better understood. 

However, it would be premature to carry out this analysis. The reason is that a sequential 
fourth generation has virtually been ruled out by precision electroweak data. Erler and 
Langacker |^ note that the S parameter is in conflict with a degenerate fourth generation 
by over three standard deviations, or 99.8%. One can weaken this discrepancy slightly by 
making the fourth generation non-degenerate, but it appears very unlikely that a sequential 
fourth generation can be accommodated, if it is the only source of new physics. One way 
around this discrepancy is to assume that there is new physics which partially cancels the 
fourth generation contribution to the S parameter (such as Majorana neutrinos, additional 
Higgs doublets, etc.). This certainly can be done, and thus searches for a sequential fourth 
generation should continue. However, this new physics will likely also contribute to b' — >■ 
b + H and to b' ^ b + Z. Thus, without some understanding of the new physics, carrying 
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out a high precision improvement of the Hon- Stuart analysis is premature. 

2. Non-chiral fermions 

Of much greater theoretical interest than a sequential fourth generation is a non-chiral 
(isosinglet or isodoublet) fourth generation. These happen automatically in a wide variety 
of models, including i^g-unification models, gauge-mediated supersymmetric models, the 
aspon CP-violation model and so on. The motivations for these non-chiral generations are 
discussed in detail in Ref. They only contribute to the S parameter at higher order, and 
are thus completely in accord with precision electroweak studies. Due to the GIM violation, 
these models have tree-level b'bH and b'bZ vertices, and thus the charged- current decay of 
the b' becomes less competitive. Without the b' ^ b + H decay, the assumption that the 
branching ratio of b' b + Z is 100% would be completely justified. 

Let us first consider the case in which b' is an isosinglet quark. The first discussion of 
b' —>■ b + H was given in 1989 by del Aguila, Kane and Quiros (AKQ) ||T0[, who looked 
at the possibility of using this decay to detect a light Higgs (if the b' mass were less than 
Mz + rrih, it would be the primary decay mode). This work was followed up by a more 
extensive analysis by del Aguila, Ametller, Kane and Quiros (AAKQ) |]lT|. A much later 
analysis of the various phenomenological aspects of isosinglet quarks can be found in the 
work of Barger, Berger and Phillips . 

Following AKQ, consider the case in which the b' only mixes with the b. The Higgs 
doublet gives the usual mass term mJjibR + h.c, as well as a term mlbib'^ + h.c. In 
addition, there are gauge invariant mass terms Myb' ib'ji + h.c. and M'b'^b^ + h.c. The 
2x2 mass matrix can then be diagonalized. The resulting mixing then gives b'bZ and b'bH 
vertices, which are proportional to m'. Thus, one gets tree level interactions, suppressed 
only by a single Cabibbo-type angle {m' /My). The angle cancels in the ratio, giving 

V{b' ^b + Z) Ml + 2M| 1^ Ml -Ml) ^ ' 
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This ratio is unity in the hmit of large Mh', and is 0.7 times the phase space factor for 
Mfyi = 200 GeV. There will also be a b'cW vertex induced by mixing, but this will be 
doubly- Cabibbo suppressed, and thus should be negligible. 

One thus sees that, once again, the b' b + H decay is comparable to the b' ^ b + Z, 
assuming the Higgs mass is not close to (or greater than) the b' mass. Again, the charged 
current b' ^ c + W decay is expected to be much smaller (and, as shown in Ref. 0, the 
b' t + W* decay will be negligible for all b' masses below 300 GeV). 

Although the isosinglet case is theoretically preferred (since isosinglet quarks automat- 
ically appear in all Eq unified models, as well as all models with a 5 + 5 of SU{5)), one 
can ask what happens if the fourth generation quarks form an isodoublet. The ratio of 
b' ^ b + H to b' ^ b + Z is the same as in the isosinglet case. However, there is one 
important difference. Although the ratio is the same, the individual rates are much smaller. 
This is because the GIM mismatch in the isodoublet case occurs in the right-handed sector, 
and there is a helicity suppression which suppresses the vertex by an additional factor of 
iTLb/My. This means that the charged current decays become much more competitive. It 
is shown in Ref. ||^ that the three-body b' ^ t + W* decay becomes competitive with the 
b' b + Z decay for b' masses of 200 GeV, and greatly exceeds it for masses above 220 
GeV. For lighter masses, the b' ^ c + W decay will still be important, and may dominate 
depending on the value of the Kb'/^fc' ratio (as in the sequential fermion case). 

3. Two-Higgs models 

In the standard two-Higgs doublet models, the so-called Model I or Model II, the Yukawa 
couplings to a Higgs are multiplied by a factor of where a is a Higgs mixing angle and 
/3 is a ratio of vacuum expectation values (depending on the specific model and the specific 
fermion charge, ^2/^1 will either be tan/5 or cot/3). In most models, Higgs mixing is fairly 
small, so cos a is near unity. In all of the above cases, this factor will change the ratio of 
b' ^ b + H to b' ^ b + Z hj a factor which is of order one. (It can't enhance the Higgs 
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decay mode too much in the sequential case, since too large an enhancement will make the 
h' or t' Yukawa coupling non-perturbative.) 



A bigger effect might be expected in Model III ||T3[. In this model, unlike Models 1 and 
II, no discrete symmetry is imposed in order to suppress tree level ffavor-changing neutral 
currents (FCNC) and thus FCNC arise, even in the sequential case. The observed lack of 
large FCNC in processes involving first-generation quarks is explained by noting that many 
models will have a FCNC coupling given by the geometric mean of the Yukawa couplings of 
the two quarks. In that case, the tree-level h'hH coupling (neglecting Higgs mixing) is given 
by g^mhrrib' / \/2Mw- 

How does this coupling affect the results? In the isosinglet case, the Model III coupling 
is of the same order of magnitude as the expected coupling induced by the GIM violation, 
and thus none of our arguments change. However, in the sequential fermion case, the Model 
III coupling is much larger than the one-loop induced h'hZ coupling. Also, in the isodoublet 
case, the Model HI coupling is much larger than the GIM-violation induced h'hZ coupling. 
Thus, since the Higgs coupling is so much larger in these two models, b' ^ b + H will 
dominate all b' decays. We conclude that in Model HI, with either a sequential or isodoublet 
b', the b' decay is dominated by b' b+H, and thus the CDF and DO bounds are completely 
inapphcable. 



4- Conclusions 

Previous searches for a fourth generation quark assume a 100% branching ratio into b+Z. 
The other neutral current decay, b' b + H, has been examined, in the sequential case, the 
isosinglet case, the isodoublet case and a two-Higgs model with tree-level FCNC. In all of 
these cases, the rate for b' ^ b + H is comparable to, or greater than, b' ^ b + Z ii the 
Higgs is kinematically accessible. 

Currently, the CDF collaboration is preparing an analysis which will give the bounds 
as a function of the branching ratio to b + Z. This analysis is conservative in that it assumes 
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that it is insensitive to other decay channels than b' ^ b + Z. However, suppose that one b' 
decays to b + Z and the other to b + H. At least one b + Z decay is needed to trigger the 
event, and the three b final state of the other b' could then be detected. The 6-tag efficiency 
in these events is expected to be considerably higher than in bbZZ events because of the 4-b 
jets final state in which at least two 6-jets have high-p^, independently of the b' and Higgs 



masses. 



1^ . This leads to the potential exciting result that the experiment could discover 



both a fourth-generation quark and a Higgs boson! 

The only discouraging model is Model HI, in the sequential or isodoublet cases. Pair- 
production of 6"s would lead to a 6-b final state, in which every b comes from a 2-body 
decay (except in the narrow region of parameter space where H W^W~ can occur). 
This would lead to quite dramatic signatures, but without a lepton trigger, finding such a 
signature would be very difficult. 

I thank the CERN Theory Group for its hospitality while this work was written, and Yao 
Yuan for her assistance. I am also grateful to Joao Guimaraes da Costa for informing me of 
the continuing interest of CDF in conducting b' searches, and for many useful discussions. 
This work was supported by the National Science Foundation. 
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